Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Locate InternalIP and ExternalIP by vSphere Network name #271

Merged
merged 1 commit into from
Nov 11, 2019

Conversation

zuzzas
Copy link
Contributor

@zuzzas zuzzas commented Oct 22, 2019

What this PR does / why we need it:

Currently, vsphere CCM will iterate over physical vNICs and setup first IP address on each as both InternalIP and . This behavior is undesirable, if we are aware of vSphere Network names.

Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged):

fixes #270

@k8s-ci-robot
Copy link
Contributor

Welcome @zuzzas!

It looks like this is your first PR to kubernetes/cloud-provider-vsphere 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes/cloud-provider-vsphere has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Oct 22, 2019
@k8s-ci-robot
Copy link
Contributor

Hi @zuzzas. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Oct 22, 2019
@zuzzas zuzzas changed the title Locate Locate InternalIP and ExternalIP by vSphere Network name Oct 22, 2019
@frapposelli
Copy link
Member

/ok-to-test
/assign @andrewsykim @dvonthenen

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Oct 23, 2019
@zuzzas
Copy link
Contributor Author

zuzzas commented Oct 23, 2019

Thanks! Going to fix tests today.

@zuzzas
Copy link
Contributor Author

zuzzas commented Oct 23, 2019

Some tests have failed, but these are not related to my changes.

@dvonthenen
Copy link
Contributor

We need to carefully think about if those new fields in the config belong in pkg/common level. I need to think about this some more, but I think might initial gut reaction is that this might be the first time we need to create an outer CPI specific config since this might only apply to the CPI.

What do you think @andrewsykim ?

@andrewsykim
Copy link
Member

My initial question is if we should accept network names or their subnets 🤔 need to noodle on that one too

@dvonthenen
Copy link
Contributor

dvonthenen commented Oct 25, 2019

My initial question is if we should accept network names or their subnets 🤔 need to noodle on that one too

If we do "VM Network" names (this should be done per VC and not in [global] also), there would be a strict requirement that each NIC must have only a single ipv4 and/or ipv6 bound to it.

@andrewsykim
Copy link
Member

there would be a strict requirement that each NIC must have only a single ipv4 and ipv6 bound to it.

So if NICs can be bound to more than 1 address it sounds like it should be based on subnet? @zuzzas thoughts?

@dvonthenen
Copy link
Contributor

dvonthenen commented Oct 25, 2019

Also, if there is only a single ipv4 or ipv6 address on the VM, we shouldn't force/require setting those new config options. It should just work.

We should also move these new config options out of pkg/common because that package is shared with CSI and these new fields have no meaning to CSI. The new config options should be placed in pkg/cloudprovider/vsphere

@andrewsykim
Copy link
Member

Fixes #266

@zuzzas
Copy link
Contributor Author

zuzzas commented Oct 28, 2019

I'll refactor the config and contribute to the discussion about network names vs subnet CIDR tomorrow.

@zuzzas zuzzas force-pushed the nameDiscovery branch 5 times, most recently from 0ccc0c3 to d746f87 Compare October 30, 2019 16:47
@zuzzas
Copy link
Contributor Author

zuzzas commented Oct 30, 2019

I've started to muse about pros and cons of L2 vs L3 matching (network names vs IPs), but the point was moot, since a user might find both approaches desirable, so I've just coded both variants.

I've also created a separate CPI-specific config, but dependency injection of it left a lot to be desired.

The tests were not updated, I'll do them once you, guys, approve of these changes.

@andrewsykim
Copy link
Member

I've started to muse about pros and cons of L2 vs L3 matching (network names vs IPs), but the point was moot, since a user might find both approaches desirable, so I've just coded both variants.

My thinking here is that because Kubernetes node addresses are strictly L3, the config we expose to users should also strictly be L3. I'm interested in the use-cases for L2 though -- is it possible to specify a network name that gives me more than 2 addresses for a node? If so what should be the expected behavior?

Copy link
Contributor

@dvonthenen dvonthenen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the update on this PR! I think we might be just about there.

pkg/cloudprovider/vsphere/config.go Show resolved Hide resolved
@dvonthenen
Copy link
Contributor

In regards to the go tests failing... I have a PR to fix one and @codenrhoden is taking a look at the other one. Tracking both in this PR #273

@dvonthenen
Copy link
Contributor

/retest

This should at least pass the CI now we have merged the two PRs to fix the unit and integration tests. Still need to resolve the overall discussion with the PR.

Copy link
Member

@andrewsykim andrewsykim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some minor comments, PR is looking great otherwise

pkg/cloudprovider/vsphere/nodemanager.go Show resolved Hide resolved
pkg/cloudprovider/vsphere/nodemanager.go Outdated Show resolved Hide resolved
@andrewsykim
Copy link
Member

/lgtm

Can you squash commits and use a more descriptive message please?

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Nov 11, 2019
You can now specify (in the new CPIConfig) "internal-network-subnet-cidr" and/or "external-network-subnet-cidr". These are used to determine which IP (the first one of each type is considered) address to map to InternalIP or ExternalIP when setting Node's object's "status.addresses" field.

Signed-off-by: Andrey Klimentyev <andrey.klimentyev@flant.com>
@k8s-ci-robot k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Nov 11, 2019
@zuzzas
Copy link
Contributor Author

zuzzas commented Nov 11, 2019

@andrewsykim
It is done.

@andrewsykim
Copy link
Member

/lgtm

Leaving final approval to @dvonthenen

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Nov 11, 2019
@andrewsykim
Copy link
Member

andrewsykim commented Nov 11, 2019

Thanks @zuzzas !

@dvonthenen
Copy link
Contributor

Seriously! Awesome contribution @zuzzas!

/approve
/lgtm

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dvonthenen

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Nov 11, 2019
@k8s-ci-robot k8s-ci-robot merged commit fca2949 into kubernetes:master Nov 11, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Both InternalIP and ExternalIP gets created for first IP on each vNIC
5 participants